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ABSTRACT 

This  paper  discusses  the  implementation  of  a  subgradient  projection 
algorithm  due  to  Sreedharan  JJdf^for  the  minimization,  subject  to  a  finite 
number  of  smooth,  convex  constraints,  of  an  objective  function  which  is  the 
sum  of  a  smooth,  strictly  convex  function  and  a  piecewise  smooth  convex 
function.  Computational  experience  with  the  algorithm  on  several  test 
problems  and  comparison  of  this  experience  with  previously  published  results 
is  presented. 
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SIGNIFICANCE  AND  EXPLANATION 


The  need  to  optimize  an  objective  function  subject  to  some  sort  of 
constraints  arises  in  almost  all  fields  that  use  mathematics.  In  many 
problems,  however,  the  objective  function  is  not  differentiable  so  that 
standard  algorithms  based  upon  gradients  are  not  applicable.  This  is  the 
case,  for  example,  with  many  problems  arising  in  economics  and  business,  and 
with  certain  design  problems  in  engineering.  One  approach  to  addressing  this 
difficulty  is  to  construct  a  suitable  generalized  gradient,  the  subgradient, 
and  base  an  algorithm  upon  it  instead  of  on  the  gradient. 

This  paper  discusses  the  implementation  of  a  subgradient  projection 
algorithm  for  the  minimization  of  a  certain  kind  of  non-differentiable, 
nonlinear,  convex  function  subject  to  a  finite  number  of  nonlinear,  convex 
constraints.  Computational  experience  with  the  algorithm  on  several  test 
problems  and  comparison  of  this  experience  with  previously  published  results 
is  presented. 
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IMPLEMENTATION  OF  A  SU&^RADIENT 
PROJECTION  ALGORITHM  II 


Robert  w.  Owens* 

1 .  Introduction 

In  the  early  1960's,  Rosen  developed  gradient  projection  algorithms  for 
nonlinear  programming  problems  subject  to  linear  [9]  and  nonlinear  [10] 
constraints.  Recent  work  by  Sreedharan  has  generalized  this  earlier  work  and 
may  be  viewed  as  the  subgradient  counterpart  of  Rosen's  work.  In  [12], 
Sreedharan  developed  a  subgradient  projection  algorithm  for  the  minimization 
of  the  sum  of  a  piecewise-affine  convex  function  and  a  smooth,  strictly  convex 
function,  subject  to  a  finite  number  of  linear  constraints.  Rubin  [11] 
reported  on  the  implementation  of  that  algorithm.  Then  in  [13],  Sreedharan 
extended  his  previous  results  to  the  case  of  an  objective  function  which  is 
the  sum  of  a  piecewise  smooth  convex  function  and  a  smooth,  strictly  convex 
function,  subject  to  a  finite  number  of  smooth,  convex  constraints.  Parallel 
to  Rubin's  investigations,  this  paper  reports  on  the  implementation  of  that 
algorithm,  since  this  latter  paper  by  Sreedharan  is  an  extension  and 
generalization  of  the  former,  the  computational  results  of  Rubin  are  relevant 
to  this  work;  consequently,  where  reasonable,  we  have  chosen  to  parallel 
Rubin's  presentation  for  ease  of  comparison  of  the  results. 

Considerable  effort  has  been  and  continues  to  be  expended  minimizing 
convex  functions  subject  to  convex  constraints  and  the  difficulties 
encountered  are  numerous.  While  no  implementable  algorithm  can  successfully 
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handle  all  problems,  Sreedharan's  algorithm  does  address  several  which  have 
plagued  others  in  the  field.  Rosen's  original  iterative  algorithms  were 
susceptible  to  what  is  sometimes  called  jamming  or  zigzagging,  i.e.  the 
generated  sequence  clustering  at  or  converging  to  a  nonoptimal  point.  Various 
techniques  have  been  proposed  to  avoid  this*  by  using  e-subgradients 
Sreedharan's  algorithm  is  guaranteed  convergence  to  the  optimal  solution. 

Others  who  have  used  e-subgradient  projection  methods,  e.g.  [3],  encounter 
computational  difficulties  since  the  complete  e-subdifferential  uses  non  local 
information  and  its  actual  determination  can  be  computationally  prohibitive. 
Sreedharan's  algorithm  requires  that  only  a  certain  subset  of  the  e-subdifferential 
be  computed,  a  more  straightforward  task.  Polak's  [7]  gradient  projection 
algorithm  essentially  projects  the  gradient  of  the  objective  function  onto 
supporting  tangent  vector  spaces,  a  method  different  than  Sreedharan's.  More¬ 
over,  Folak  hypothesizes  a  certain  linear  independence  of  the  gradients  of  the 
e-binding  constraints  whereas  the  algorithm  we  are  reporting  on  requires  no 
such  assumption. 


2.  Problem 


Let  ft  C  R  be  a  nonempty,  open,  convex  set,  and  let  f,  g^,  v^, 
i*1 ,2, . . . ,m,  j«1,2,...,n  be  convex  differentiable  functions  from 
ft  into  R.  Also,  let  f  be  strictly  convex.  Let  X  =  {x  e  ft|g  (x)  <  0, 
i=1,2, . . . ,ra} .  He  assume  that  X  is  bounded  and  satisfies  Slater's  constraint 
qualification:  there  exists  a  e  X  such  that  g^a)  <  0,  i“1,2,...,m.  Let 

v(x)  *  max{v^(x J [ j=1, 2, . . . ,n} .  The  problem  to  be  solved,  referred  to  as 
problem  (P)  is: 

minimize:  f(x)  +  v(x) 

(P): 

subject  to :  x  e  X 

Note  that  (P)  has  a  unique  solution  since  f  +  v  is  continuous  and 
strictly  convex  on  the  compact  set  X.  Note  also,  however,  that  in  general 
f  +  v  is  not  differentiable. 


r 


3.  Notation 

Given  x,  y  e  Rd,  we  denote  the  standard  Euclidean  inner  product  of  x 

and  y  by  juxtapostion,  i.e.  xy,  and  the  corresponding  length  of  x  is 

d 

denoted  by  Ixl .  Given  a  nonempty,  closed  convex  set  SCR,  we  denote  by 
N[S]  the  unique  point  of  S  of  least  norm,  y  *  N [S]  is  the  projection  of 
0  onto  S  and  is  characterized  by  the  property  that  y(x-y)  >  0  for  all 
x  e  S.  For  a  differentiable  function  $,  denote  by  $'(x)  the  gradient  of 
$  at  x.  Given  any  set  S  C  Rd,  let  cone  S  be  the  convex  cone  with  apex 

0  generated  by  S  and  conv  S  the  convex  hull  of  the  set  S. 

For  any  point  x  e  X  and  any  e  >  0,  define  the  following  four  sets: 

I£(x)  =  {i | g^(x )  >  -e,  1  <  i  <  m} 

J£(x)  =  { jlv^(x)  >  v(x)  -e,  1  <  j  <  n} 

C£(x)  «  cone  {g^(x)|i  e  I£(x)} 

K£(x)  *  conv  { v^ ( x) | j  e  J£(x)}. 

With  x  e  X,  IQ(x)  is  the  set  of  active  (i.e.  binding)  constraints  at  x, 
while  I£(x),  with  e  >  0,  is  the  set  of  e-active  (i.e.  almost  binding) 
constraints  at  x.  J£(x),  with  e  >  0  and  x  6  X,  is  similarly  interpreted 
in  terms  of  the  maximizing  functions  that  define  the  function  v.  ce(x)  and 
K£(x)  are  the  easily  computable  subsets  of  the  e-subdifferentials  referred 


to  in  section  1 


Step  1 i  Carry  out  an  unconstrained  minimization  of  f .  If  no  minimizer 
exists  in  ft,  go  to  step  3;  if  the  minimizer  c  exists  in  ft  but  c  ^  X, 

go  to  step  3.  If  c  8  X,  proceed  to  step  2. 

Step  2;  If  v^(c)  •  0  for  j*1,2,...,n,  then  STOP:  c  solves  (P). 

Otherwise  proceed  to  step  3. 

Step  3 :  Start  with  an  arbitrary  xQ  e  X,  6  >  1,  and  set  k  *  0.  Let 

0  <  <  -max{g^( a) | 1  <  i  <  m) .  Set  e  *  e^. 

Step  4:  Compute  yQ  -  Nlf'tx^)  +  l^tx^)  +  C^x^)].  If  yQ  *  0, 

STOP;  xk  solves  (P).  Otherwise  proceed  to  step  5. 

Step  5:  Compute  yc  *  Ntf'tx^)  +  K. ( x^ )  +  W]* 

2 

Step  68  If  ly£l  >  e,  then  set  e^  *  e,  Sfc  *  y£,  and  go  to  Step  8. 

Step  7:  Replace  e  by  e/6  and  go  to  step  5. 

Step  8:  Let  I  *  I£  (xfc).  If  I  *  ♦»  then  let  ”  0  and  *  0. 

k 

If  I  *  <t,  then  proceed  as  follows.  Let  Y^  *  g^fx^Jg’ 
and  solve  the  following  linear  programming  problem  for  >  0,  i  e  I: 

Minimize:  J  u. 

iei 

Subject  to:  l  Y^H.  >  <91  (x.  >*»  j  e  I. 
iei  3  3 

>  o,  iei 


(x^  for  i,j  e  I 


Set  -  l  and  Mfc  “  ( I f *  (x^ )l  +  max  lv’ ^ ) I  Jlu^l 

.2 


Step  9:  Let  Xfc  -  IS^I  /(2M)t  +  1)  and  t^ 


sk  *  xkV 


X.  -  ISJ 
k 

Step  10:  Pind  <*k  »  maxto^  -  at^  e  X  and  g^x^  -  at^)  <  g^x^)  vi  * 


Step  lit 

Find 

e  [0,a^3  such  that  there 

exists 

%  •  f,,\  - 

w 

*  V*k  ’  W  *lth  Vk  ■ 

0.  If  no  such  a 

k 

exists. 

then  set 

"  V 

Step  12: 

Set 

S.1  •  %  •  SS*  lnct“*nt 

k  by  1,  and  go  to 

step  4. 

Steps  1  and  2  are  present  for  technical  reasons  and  rule  out  the 
possibility  that  a  given  problem  has  an  easily  obtained  solution.  For  many  of 
the  test  problems  considered,  these  steps  were  bypassed  since  a  specific 
starting  point  was  desired  or  it  was  known  that  a  trivial  solution  did  not 
exist.  In  the  fully  implemented  version  of  this  algorithm  actually  tested, 
step  1  was  performed  by  calling  the  ZMSL  subroutine  ZXCGR  which  employs  a 
conjugate  gradient  method  to  find  the  unconstrained  minimum  of  an  n  variable 
function  of  class  C1. 

The  stopping  criteria  of  either  step  2  or  step  4  would,  in  practice,  be 
replaced  by  lyQl  or  lv^(c)l  becoming  sufficiently  sauill. 

In  [13]  it  is  shown  that  -t^,  from  step  9,  is  a  feasible  direction  of 
strict  descent  for  f  +  v  and  that  a^,  from  step  11,  is  strictly  positive. 

Before  describing  computational  trials  on  test  problems,  the  more 
substantial  subproblems  involved  in  carrying  out  the  steps  of  the  given 
algorithm  are  discussed.  Specifically,  we  mention  the  projection  problem 
inherent  in  both  steps  4  and  5,  the  linear  programming  problem  of  step  8,  the 
line  search  required  in  step  10,  and  the  one  dimensional  minimization 
indirectly  called  for  in  step  11. 


-6- 
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5.  Subproblems 

The  problem  of  computing  the  projection  of  0  onto  f' (x^)  + 
with  e  *  0  ( step  4 )  or  £  >  0  ( step  5 )  is  solved  almost  exactly  as  in  [11], 

i.e.  the  problem  is  expressed  as  a  quadratic  programming  problem  which  is  then 
solved  by  standard  techniques.  For  completeness  we  outline  the  key  ideas?  for 


vv  *  w 


details  see  Rubin  [11]  and  Cottle  [4]. 

Assume  that  x  e  X,  e  >  0,  I  (x)  -  {i,...,i  }  t  and 

e  *  P 

J  (x)  *  y  ■  N[f'(x)  +  K  (x)  +  C  (x)]  can  be  written  as 

€  1  q  €  € 

y  »  f'(x)  +  f  ol  g*  (x)  +  f  $  v*  (x),  where  a  >  0,  k-1,...,p, 

k-1  *  *k  k-1  *  3k 

6.  >  0,  k  -1,...,q,  \  0.  -  1,  and  (1/2)lf'(x)  +  £oi  g!  (x)  +  £0,v  (x)I2 

k  k-1  *  k  3k 

is  a  minimtmt  over  all  such  and  0^,  s.  Let  h^  -  V  (x)  +  v*  (x). 


k-1, ... ,q.  Then  f'(x)  +  jf  0.v' 

k- 


(x) 


0^h^.  let  e  —  (0,...,0,1,...,1) 


Jk  k-1 

be  the  p  +  q  vector  the  first  p  of  whose  components  are  0  and  the  last 

q  of  whose  components  are  1.  Let  u  -  $^,...,0^), 

M  ■  (g'  (x) «... ,g'  (x),  h  ,...,h  ),  and  Q  -  MTM,  where  the  superscript  T 
i1  P  1  q 

denotes  the  transpose.  The  projection  problem  under  consideration  can  now  be 


expressed  as  the  quadratic  programming  problem: 


Minimise:  1/2  uQu 

Subject  to:  ue  -  1 
u  >  0 


Rubin  adapted  an  algorithm  of  Wolfe  to  solve  this  problem,  an  algorithm  which 
is  guaranteed  to  terminate  at  the  solution  in  finitely  many  steps.  In  the 
present  work,  a  package  1]  develo  *d  at  the  University  of  Wi scons in-Madison, 
based  on  Cottle's  version  '  .  the  principal  pivoting  method  [4]  was  used. 
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Since  the  primary  means  of  reporting  on  the  success  of  the  subgradient 


projection  algorithm  is  to  give  the  number  of  iterations  needed  to  obtain  a 
fixed  accuracy,  little  extraordinary  effort  was  expended  optimizing  the 
computations  involved  in  each  iteration.  Instead,  IMSL  subprograms  were  used 
where  convenient. 

Step  8  requires  the  solution  of  the  linear  programming  problem  stated  in 
section  4;  this  problem  is  the  dual  of  a  standard  linear  programming  problem 
for  iriiich  subroutine  ZX3LP,  a  "revised  simplex  algorithm  -  easy  to  use 
version,"  is  immediately  applicable.  No  difficulties  were  encountered. 

Step  10  involves  a  line  search  to  locate  a^,  the  maximum  a  such  that 
x^  -  at^  e  X  and  no  previously  e -binding  constraint  becomes  more  binding. 
Letting 


Gt(a) 


gi(xk-°V  M1 

9i(xk  "  “V  ‘  *i(V  1  e  1 


and  G(a)  =  max{G^(a) | 1  <  i  <  m} ,  and  using  the  convexity  of  the  g^,  step 
10  can  be  rephrased  as:  find  the  unique  positive  root  of  G(a)  =  0. 

After  locating  0  <  <  a2  such  that  Gfc^)  <  0  <  G(<*2),  this  latter 

problem  is  solved  by  bisection  with  the  stopping  criteria  being  that 
0  <  G(c)  <  10-12  and  \a^  -  o2l  <  10~12. 

Step  11  requires  that  we  locate,  if  it  exists,  e  [0,a^]  such  that 
there  exists  e  f’tx^  -  a^t^)  +  K0  -  a^t^)  with  z^t^  =  Of  if  no  such 
exists,  then  set  a^  *  a^.  This  is  equivalent  to  locating  the  minimum  of 
the  strictly  convex  function  F(a)  ”  f(x  -  at  )  +  v(x  -  at.)  on  [0,a  ]> 

a  K  J\  K  XI 


see  [13,  Lemma  5.22].  The  IMSL  subroutine  ZXGSP  for  one-dimensional  unimodal 


Ad 


function  minimisation  using  the  Golden  Section  search  method  was  used  to  solve 

this  subproblem.  The  stopping  criterion  employed  was  that  the  interval  within 

-12 

which  the  minimum  occurs  should  be  less  than  the  minimum  of  10  and 
10  *  a)c*  Occasionally  an  error  message  appeared  saying  that  the  function 

did  not  appear  unimodal  to  the  subroutiner  this  was  clearly  due  to  roundoff 
errors  becoming  significant.  Nevertheless,  these  warnings  posed  no  major 
difficulties. 

With  the  earlier,  linear  version  of  Sreedharan's  algorithm,  Rubin  [11] 
was  able  to  carry  out  an  efficient  search  for  without  explicitly  locating 

by  exploiting  the  linearity  of  the  g^  and  v^  functions.  Although 
some  improvement  for  finding  in  the  present  implementation  is  undoubtably 

possible,  no  such  dramatic  economies  are  to  be  expected  due  to  the 
nonlinearity  of  the  functions  involved. 


rotational  Trial* 


6.  Oomg 

The  subgradient  projection  algorithm  wee  implemented  in  Fortran  77  on  a 

VAX  1 1-780  ueing  double  precieion  (12-13  digits  accuracy)  and  employing  the 

software  described  in  section  5.  For  test  purposes ,  6  (step  3)  was  taken  to 

be  10.  In  order  to  avoid  taking  unnecessarily  small  steps  early  in  a  problem, 

e  was  reset  to  eQ  at  step  4  for  the  first  4  iterations  and  was  allowed  to 

follow  the  given  nonincreasing  pattern  thereafter.  The  convergence  criterion 

-12 

of  step  4  was  relaxed  to  produce  termination  if  either  ly^l  <10  or 

-  Vt'  < 

The  first  two  problems  below  are  ones  considered  by  Rubin  [11],  neither 
of  which  meet  the  hypothesis  that  f  be  strictly  convex  since  in  each 
f  -  0. 

Example  1.  The  first  example  was  proposed  by  Wolfe  [14]  and  uses  d  *  2, 

f  =  0,  v1(x,y)  ■  -x,  v2(x,y)  -  x  ♦  y,  v3(x,y)  -  x  -  2y,  n  ■  3.  Following 

Rubin,  we  investigated  cases  with  constraints  leading  to  a  rectangular 

feasible  region  having  the  global  minimirer  (0,0)  of  f  +  v  interior  to,  on 

the  boundary  of,  and  exterior  to  the  feasible  region.  Table  I  summarizes  the 

results.  The  lower  left  and  upper  right  corners  of  the  feasible  region  are 

given,  as  is  the  point  at  which  the  optimum  value  is  achieved  and  the  nisaber 

of  iterations  required  to  obtain  termination.  In  each  case 

e  -  -1/2  max  g  (x  ,y ). 

1<i<4 

The  slow  convergence  occurs  only,  but  not  always,  in  cases  where  the 
optimal  solution  is  at  a  corner.  The  linear  version  of  this  algorithm  was 
able  to  handle  all  cases  in  at  most  2  iterations  [11]. 


I 

■v 
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Lower  Left 
Corner 


Upper  Right 
Corner 


Solution 


Number  of 
Iterations 


(-1,-1) 

<  1,  D 

(  0,  0) 

0 

(  1,-1) 

(  2,  1) 

(  1,  0) 

1 

(-3,  1) 

(  2,  3) 

(-1/2,1) 

2 

(-2,-1) 

(-1,  D 

(-1,  0) 

1 

(-2,-4) 

(  2,-2) 

(-2,-2) 

18 

(  1,  2) 

(  3,  4) 

'  1,  2) 

13 

(-2,  1) 

(-1,  2) 

(-1,3/2) 

1 

(-4,-3) 

(-1,-1) 

(-1,-1) 

17 

(  0,  0) 

(  1,  D 

(  0,  0) 

1 

(  0,-1) 

(  1,  D 

(  0,  0) 

1 

Table  I 

Test  results  for  example  1. 

Example  2.  The  second  example,  attributed  to  Powell,  also  appears  in  [14]. 

In  that  paper,  an  unconstrained  minimisation  algorithm  proposed  by  Wolfe 
achieves  only  linear  convergence  when  applied  to  this  example,  d  ■  2, 
f  =  0,  n  -  5  with  vi(x,y)  -  (cos  — g— )x  ♦  (sin  -g-)y  for  i«1,...,5.  We 
set  X  -  conv{(±1,  ±1)},  so  that  the  global  minimixer  (0,0)  is  interior  to 
X,  and  we  choose  the  initial  point  (p  cos-  ,  p  sing-  )  for  p  >  0  as  ms 
considered  by  both  Wolfe  and  Rubin.  Computational  results  for  various  values 
and  ranges  of  p  are  summarised  in  table  II.  p  <  sec  g  ■  1.23605  yields  an 
initial  point  inside  of  X  while  p  >  sec  g  puts  xQ  outside  of  X. 


-11 


Rubin  also  tested  the  same  function  f  +  v  with  the  feasible  region 


determined  by  x,  y  <  0  and  x  +  y  <  -1  and  found  convergence  in  2  to  4 
iterations  from  various  starting  points.  Our  results  are  in  basic 
agreement.  We  had  to  start  quite  far  out  in  the  third  quadrant,  e.g. 
(-10,-10)  before  the  algorithm  required  5  iterations  to  terminate. 

p  No.  of  Iterations 


[0,  sec  1 

(sec  |,  32.94)  2 

(32.95,  100]*  3 

1234  10 


Table  II 

Test  results  for  example  2. 


*  100  was  the  largest  value  tested  in  this  range. 


Example  3.  A  third  example  tested  comes  from  [2,  p.76]  where  the  problem  is 
solved  by  a  parametric  feasible  direction  (PFD)  algorithm  -  section  7  of  the 
book.  The  problem  involves  a  quadratic  objective  function  in  5  variables  and 
7  constraints  involving  linear,  quadratic,  and  exponential  terms;  v  =  0 
since  their  algorithm  is  not  designed  to  handle  nondif ferentiable  functions. 
Our  results  are  in  qualitative  agreement.  The  PFD  algorithm  terminated  after 


35  iterations  with  6  decimal  figures  of  accuracy;  the  subgradient  projection 


algorithm  terminated  after  41  iterations  with  9  decimal  figures  of  accuracy. 

Example  4.  The  fourth  example  tested  was  introduced  by  Demyanov  and  Malozemov 

(5]  to  illustrate  jamming  of  a  certain  subgradient  optimization  algorithm.  Zn 

this  example,  d  *  2,  f(x,y)  =  1  (not  strictly  convex),  n  »  3  with 

2  2 

v1(x,y)  »  -5x  +  y,  v2<x,y)  ■  x  +  y  +  4y,  v3<x,y)  -  5x  +  y.  He  also 

2  2 

impose  the  artificial  constraint  g(x,y)  *  x  +  y  -  99  <  0.  Starting  from 

2  2  17 

any  initial  point  on  the  circle  (x  +  5/2)  +  (y  ♦  3/2)  ”  ~,  with 

-1.928  <  x  <  -1.831  and  y  >  0,  at  which  v,  and  v2  are  maximands  for 

v,  Demyanov  and  Malozemov  note  that  the  usual  subgradient  optimization 

algorithm  converges  to  the  nonoptimal  point  (0,0),  not  to  (0,-3)  as  it  should. 

When  our  subgradient  projection  algorithm  was  used  to  solve  the  above 

problem  with  nine  different  starting  point  satisfying  the  above  conditions, 

the  optimum  solution  was  obtained  in  at  most  4  iterations.  For  various  other 

starting  points,  each  of  which  had  v^  and  v2  maximands  for  v,  the 

algorithm  required  from  1  to  7  iterations  before  termination.  In  all  cases, 

-2 

eQ  **  10  was  used. 

Example  5.  The  fifth  example,  appearing  in  [1,  p.447],  has  no  optimum 
solution,  but  a  straightforward  application  of  a  steepest  feasible  descent 
direction  method  without  "anti-zigzagging"  precautions  leads  to  convergence. 
The  problem  is 


4/2  2\3/4 

Minimize:  f(x,y,z)  ■  —  (x  -xy+y)  -Z 

Subject  to:  x,y,z  >  0 

and  with  the  initial  point  specified  to  be  (0,  1/4,  1/2).  Clearly  f(0,0,z), 
z  >  0,  is  negative  and  arbitrarily  large  in  magnitude  for  appropriately 


chOMn  z.  It  is  easily  shorn  that,  when  the  steepest  feasible  descent 

direction  aethod  is  employed,  the  iterates  (x^,  y^,  z^)  converge  to 

(0,0,  with  the  third  coordinate  aonotonically  increasing.  For  test 

2  2  2 

purposes,  we  introduced  the  extra,  artificial  constraint  x  ♦  y  +  z  <  225 
in  order  that  the  problea  have  a  solution,  albeit  one  very  far  reaoved  from 
the  "false  solution"  to  which  the  simpler  algorithm  converged.  By  k  =*  5  the 

4W2 

third  coordinate  had  exceeded  — — j  by  k  ■  82  the  artificial  constraint  was 

a 

binding;  and  the  algorithm  successfully  terminated  after  111  iterations. 

example  6.  Finally  the  algorithm  was  tested  on  a  family  of  problems  each 

involving  two  variables  (d  *  2),  three  v^s  (n  -  3),  and  four  constraints 

2  2 

(m  -  4).  All  functions  were  quadratic  of  the  fora  a.(x-a_)  +  a,(y-a.)  -  a,, 

12  3  4  D 

and  various  trials  were  conducted  with  different  choices  of  the  parameters 

a^.  For  example,  consider  the  two  problems  with  Vj(x,y)  “  (x-2)  +  y, 

v2(x,y)  -  (1/2)(x2  ♦  y2),  v3(x,y)  -  x2  ♦  (y-2)2,  g^x.y)  -  (x-1)2  ♦  y2  -  4, 

g2(x,y)  *  (xtl)2  ♦  y2  -  4,  g3(x,y)  *  x?  ♦  (y-1)2  -  4,  g4(x,y)  »  x2  +  (y+1)2  -  4. 

With  f(x,y)  -  (x-2)2  +  (y-2)2  the  solution  (a, a),  a  -  ,  was  found  in 

2  2 

1  step,  while  with  f(x,y)  ■  (x-4)  +  (y-1)  the  same  optimum  point  was  found 

in  9  iterations.  In  each  case  Xq  -  (0,0)  was  obtained  by  the  fully 
implemented  algorithm  and  -  1.5. 
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7.  Conclusions 


As  Rubin  [11]  found  with  the  earlier ,  linear  version  of  this  algorithm, 
the  subgradient  projection  algorithm  is  a  viable  method  for  solving  a  number 
of  convex  programming  problems  including  some  which  do  not  meet  the  hypotheses 
needed  to  guarantee  convergence,  e.g.  examples  1,2,4.  When  applied  to  smooth, 
ie.  differentiable,  problems,  e.g.  example  3,  it  appears  to  perform  competi¬ 
tively  with  less  general  algorithms  designed  specifically  to  solve  such 
problems.  It  successfully  avoids  jamming  on  test  problems  that  have  been 
specifically  designed  to  jam,  e.g.  example  4.  Zt  qualitatively  replicates  the 
results  of  the  earlier  linear  method  when  applied  to  previously  tested 
problems.  And  it  successfully  solves  in  a  small  number  of  iterations  larger 
test  problems  designed  to  exercise  the  full  range  of  capabilities  built  into 
the  algorithm. 

Since  one  always  wants  faster  convergence  than  is  available,  it  might  be 
worth  speculating  on  what  factors  might  limit  the  performance  of  this 
algorithm.  The  anti- jamming  techniques,  although  clearly  necessary,  seem  to 
entail  a  rather  heavy  cost  in  slowing  down  performance.  Whereas  the  gradient 
and  even  the  subdifferential  are  local  entities,  the  e-subdif ferential  is 
not.  While  many  e-subgradient  based  algorithms  require  knowledge  of  the 
entire  e-subdifferential,  a  formidable  if  not  prohibitive  task,  Sreedharan's 
algorithm  requires  only  an  easily  computable  subset  of  that  set.  An  appro¬ 
priate  element  of  the  e-subdifferential  is  chosen  and  used  to  construct  a 
feasible  direction  of  strict  descent  while  avoiding  jamming.  One  possible 
source  of  difficulty  is  the  difference  between  the  directional  derivative 
F' (x;u)  and  the  e-subgradient  based  approximation  to  it,  say  F^(x»u), 
where  F  is  the  objective  function,  e.g.  F  ■  f  +  v  in  our  case.  Hiriart- 
Urruty  [6]  has  found  that  |F'(x;u)  -  F^(x;u)|  ■  0(/e)  as  e+0.  In  some  of 


our  trials,  indirect  evidence  of  this  qualitative  behavior  can  be  found 
indicating  a  possible  culprit  when  slow  convergence  is  observed.  Consider  the 
following  simple  problem: 

2  2 

Minimize:  F(x,y)  *  ( x-3 )  +  (y-1) 

4  4 

Subject  to:  x  +  y  <  t 

Referring  to  Table  III,  the  correspondence  either  between  and  the  number 

of  correct  figures  of  the  iterate  or  between  and  lyoi—  suggests  an 

/e -order  convergence  rate  in  line  with  Hiriart-Urruty ' s  result. 


k 


(xk,yk) 


iy„i 

'0  • 


0 


5 


10 


15 


20 


(0,  0) 

(.970,  .58) 

(.97122,  .5762395) 

(.97121490,  .5762462) 

( .97121493596,  .5762460176) 

(.9712149358190,  .5762460177155) 


.5 


5  x  10 


5  x  10“ 


5  x  10“ 


5  x  10 


10 


13 


16 


6 

2.67  x  10“2 
4.38  x  10"5 
1.08  x  10"6 
1.13  x  10"9 


Table  XIX 


Indirect  or  suggestive  evidence  of  a  /e -order  rate  of  convergence 
subgradient  projection  algorithm 


*  The  actual  solution  correct  to  12  digits. 


1 


r(xk'yk) 


10 


4.29559 


4.29553627388 


4.2955362736910 


4.295536273690917 


of  the 
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